The Dreaded Halo - an advanced focus stacking tutorial
/Ask ten macro photographers what spoils more of their pictures than anything else, and eleven of them will give you some variation on the theme, “halo”. And this should not come as a surprise to anyone who has been focus stacking at magnification for any amount of time. The cause is multifactorial, so the cure must also be multimodal, and it is. It requires a good understanding of the cause, and a strong working knowledge of the various tools available in your stacking software. Getting rid of halo artifact starts before the first shot is taken and is not complete even after the final image has been saved from the stacking program. Confused?
With the help of a few examples I will walk you through my own technique for avoiding and/or correcting halo in my focus stacked images. It is not quick or simple, it involves a lot of steps, and requires a fair amount of judgement. Improvement is usually gradual, as the photographer becomes more familiar with the tools, more intentional in selecting settings, and more deliberate in planning a halo-limiting workflow for each individual focus stacking project.
What is halo?
Halo has become something of a catchall term for any haziness, softening, loss of contrast, or indistinct detail seen around foreground (usually) elements in the final photograph. So the term halo can mean several different things depending on the context. Some of these changes are virtually impossible to prevent, but can be significantly improved with some of the measures I will describe later. The majority are of one particular type and these “lost detail” halos are going to be the primary focus of this discussion. Look at the image below.
The Dmap output from Zerene Stacker (ZS)
This is the depth map output from a 62-image stack that was processed using a sub-stacking, or slabbing routine in Zerene Stacker¹. Nothing has been done to this image, either prior to export from Lightroom (LR) to ZS, or after export back to LR. Let’s take a closer look at the left fore-appendage, also from the Dmap output.
Halo artifact clearly seen along top border of left fore-appendage (and around “knee”)
A crucial point to note is that the Pmax (pyramidal algorithm) looks quite different.
Note the different character of the halo in the Pmax output
Understanding of this difference will inform much of what we must do to avoid or remove halo, but first we should understand what we are looking at in the first cropped image. The halo that we see is actually real, a part of the image that has been faithfully reproduced in the images before stacking. Think of it like this, when the appendage closest to the camera is in sharp focus, the one in the distance is a soft blur because it is still out of focus. ZS has no trouble selecting the sharp parts of the images and forming a well demarcated composite of the appendage closest to us. As the stack progresses we soon get to the appendage in the relative background and it also starts to fall into focus. But the near appendage has not disappeared - it is still in the foreground, only now it is out of focus, and appears as a soft, enlarged, indistinct version of the leg. Because of the way we are positioned there are some parts of the more distant appendage that are never imaged without the defocused foreground appendage being directly in the way - and that is exactly what our camera captures. It faithfully reproduces the background appendage through the out of focus foreground one. It is characteristic of the depth mapping process to show us this transition zone and that is why the Dmap output is usually where the halo is most prominent.
The Pmax output uses a different set of computations that usually result in an image that shows enhanced contrast, at the expense of flatter colors and a loss of textural detail, but crucially, with significantly less of this type of halo. And it is this key difference that offers us an opportunity to lessen the impact of the halo. But before we get to that, this information also provides a few hints to how we may be able to plan an image to minimize the likelihood that this artifact will form in the first place. Let us turn our attention to the area where the left second appendage overlays the right third appendage.
Significantly worse halo around left second appendage
Looking only at the halo (the foreground transparency is a subtly different artifact that I will cover separately), we can see it is significantly worse in this area. Why? The size of the halo zone is proportional to the distance separating the fore- and background elements of the image. It is also true that the perceived density of the blur is inversely proportional to the distance and this becomes relevant when we look at the Pmax image in this same area.
Ignoring the transparency, the Pmax shows even less halo artifact
It is this discrepancy in how halo manifests with differing separation of foreground and background, that gives us our first strategy for managing the problem.
Avoiding halo
Obviously, when we have overlapping elements, we are going to have halo, so to avoid all halo we have to take pictures without overlapping elements. While this sounds a little facetious on the surface, it is actually a crucial part of my pre-shot planning. I will examine my subject from every conceivable angle, after pinning, to see if there is a way to minimize overlap, and there almost always are options that will greatly reduce the severity of the problem. True, it often involves repositioning elements while adjusting perspective, but it can be done in most cases. A word of caution; be careful to avoid putting your subject in an unnatural pose - it is easy to become so focused on eliminating artifact that you end up with a pristine image of an entirely unbelievable subject!
Sometimes, of course, the nature of the subject makes avoidance of all overlap impractical or impossible. In these cases we need to shift our attention to damage control and find ways to lessen the impact of the artifact and make subsequent correction both simpler and more effective. I do this by examining each point of overlap and seeking to either minimize or maximize overlap separation. When the overlapping elements are quite close together as in the forelimb example above, retouching the image is less likely to resolve the halo completely, but the area that will need post-stack editing will be small. When the separation is large, the artifact is usually easily removed during retouching. It stands to reason, then, that separations that are between the two extremes are the most difficult to correct completely. And that is what I try to do - bring the crossing elements as close together or move them as far apart as the subject will allow while keeping the final image as natural and lifelike as possible. This takes an enormous amount of practice and thoughtful experimentation, but it will come to those who have the determination to stay with it. But no matter how meticulously we work to avoid the dreaded halo, it can never be completely banished, so we also need to know how to correct what remains.
Removing halo
We have photographed and focus stacked this tiny weevil and it is only after all that work has been completed that we notice all that halo around the appendages - our opportunity to avoid the problem has passed, as has the chance to minimize what we need to do next. Now we need to change gears and address removal of the halo left behind.
This is the point at which many an inexperienced macro photographer makes a decision that seals the fate of the image. Looking at the two outputs (you did run both algorithms, didn’t you?) the photographer sees a Dmap replete with halo and a less attractive Pmax that has little or no halo, but little or no textural detail and colors that are flat and unappealing. So the Dmap goes in the trash and the Pmax is saved as the final image and it is time to move on to the next stack to repeat the same mistake. Eventually, many of these photographers will ask me why I recommend they always run both algorithms - they will tell me they have been doing it for years and the Dmap is always horrible and ends up in the trash - why not just run Pmax and be done with it?
Rarely, if ever, does the Dmap or Pmax output become my final image. Put another way, the best images are almost always made up from the best parts of both the Pmax and the Dmap outputs. The Dmap image provides the tonal complexity and richness of the image while preserving the subtle surface textures that make the picture come alive. The Pmax algorithm is designed to pay less attention to those characteristics of the image and instead ensure that complex detail, especially complex detail where fine elements are crossing one another, remains intact. And the best images give us both. It just so happens that these two sets of of priorities are complementary, almost perfectly so, such that we can, with care and practice, preserve the best of both worlds in each image we make. We do this through the process called retouching.
In retouching we select a baseline image and use any other images available to us to correct any of the predictable shortcomings in the baseline image. It is the nature of the two algorithms that the good stuff in a Dmap output is fairly diffuse and evenly distributed, while the good stuff from the Pmax is discrete and focal. It stands to reason then, that the Dmap should be selected as the baseline image and the problematic areas are replaced with their better defined counterparts from the Pmax image. This is not to say that you couldn’t do it the other way around, you certainly could, but it would take a lot more time and work and would be far more likely to result in overlooked areas. Our example above is fairly typical and the Dmap is clearly the best option for a baseline.
A vital point that I will mention briefly, but am not going to get into at depth in this article, concerns the options that are available of adjusting the way that ZS performs the Dmap calculations. How we set the Estimation Radius and Smoothing Radius prior to the Dmap operation significantly affects the character of the output, including that way in which halo is manifest. Also crucial is the selection of a contrast threshold, during the Dmap operation, which itself is directly influenced by the setting we choose for the ER/SR. For images that lack a lot of fine crossing detail, a higher ER/SR will allow us to set a lower contrast threshold, and will help eliminate some of the artifact. But not every image will have the characteristics to allow this and a judgement call is required. The sweet spot, when we find it, will minimize artifact and greatly reduce the need for aggressive retouching
The way retouching works is mostly science and partly magic, but completely necessary. We select the baseline image and this becomes our canvas. Then we select another image, usually but by no means always, the Pmax output, and this becomes our brush. As we paint onto the baseline image the pixels in that image are replaced by those from the second image. It is not nearly that simple and there are several different brushes that perform slightly different functions, but this is still a helpful way to think about what is happening. Some of the advanced retouching “brushes” will compare the baseline pixel to the corresponding pixel from the second image and make the change only if certain conditions are met - the pixel will only be swapped if the source pixel is lighter, say, than the destination pixel ( the lightening brush works this way), otherwise the replacement does not happen. But 90-95% of the time, the details brush in ZS is all you will need to get the results you are looking for. I would tell you how this brush works, if I knew, but alas, I don’t. What I do know is that it works, and usually it works flawlessly.
A word about the other focus stacking programs is called for here. If you want to know is sordid detail how ZS compares to the other popular stacker, Helicon, read my three part essay on the subject, but it boils down to one crucial difference - only ZS has a complete set of retouching tools (at this writing). It is as simple as that. The tool that is absent from the Helicon program is one that may seem trivial and insignificant, or even totally irrelevant if you don’t believe in retouching (in which case this argument will not sway you), but is one that I use in virtually every focus stacked image that I produce. It is a tool that is as basic to my retouching workflow as a camera is to my photography. This is the ability to stack a subset of my original images and produce an output that remains in the current project and therefor may be used as a retouching source. This is not the case with Helicon - you can run a selected sub stack and produce a new output, but that output becomes part of a different project and cannot be used as a retouching source in the original project. The reason is because the substack has undergone a different alignment than the original source images and will not “fit” over the baseline retouching target without realignment - which then renders the original input images misaligned. You cannot retouch from the original source images and the substack outputs within the same active project. In Zerene Stacker you can. Quod erat demonstrandum.
Back to the retouching… If we have taken our photographs to avoid the middle of the road overlaps as much as possible, it is usually a simple matter to paint away the halo in situations whrere the separation is greatest - it just vanishes with a stroke of the brush. Where separation is minimal and there is some residual halo seen in the Pmax image, the retouching does not usually remove the halo entirely, but it will often dramatically decrease the overall impact. Here is my weevil after a very quick and basic retouch, using only the details brush
Dmap retouched using Pmax only
You will see the forelimb halo is almost completely gone and the halo around the second appendage is completely gone. The retouching has improved but not completely resolved the transparency in the area of the second appendage. The astute observer will notice a marked improvement in the contrast of the nearer of the two antennae, but it is not perfect. To remove the residual transparency and clean up the appearance of the antenna, I ran sub stacks of each of the two problem areas and retouched the same Dmap baseline image using the outputs from these two selected stacks (this is what you cannot do in the other program). And this is the result.
Antenna and transparency correct
I used the darken brush to correct the transparency, retouching from a new sub stack of that appendage and corrected the antenna with the lighten and details brushes. However, I am not completely satisfied with the antenna as there is still a tiny amount of indistinct halo around the final segment. There is also some vague residual halo around the upper edge of the left first appendage. Retouching has vastly improved both of these areas but not enough to call the image complete. There is one more step to complete the image.
Completing the image
Make that two steps. It is my practice not to touch the input images in LR before taking them into ZS to stack and retouch. This is because all my images are deliberately slightly underexposed to allow for the artificial exaggeration of the dynamic range that occurs during TIFF conversion and again during Pmax stacking. To prevent specular highlights I expose my images slightly to the left on the histogram and am happy to accept a little clipping in the darkest areas, as this detail can usually be recovered during this next step. One important note is that this technique of slightly underexposing my images also tends to minimize the amount of halo-like artifact in the Pmax outputs, further augmenting the quality of the post-retouch image. Once my retouched image is saved back to LR it is finally time to do my basic edits. This used to be something I did prior to stacking, but have found this order to be far more effective and yield much more pleasing final images.
This edit is minimal and usually involves adjusting the highlights down a fraction and the shadows up a little. I set my white and black points as usual and then bump the details up a little. I will dehaze sparingly if needed, but keeping an eye on the exposure to avoiding clipping the shadows. Lastly, I may nudge the vibrance up, but will balance this by reducing the saturation an even smaller amount.
Finished in LR it is time for the obligatory visit to the world of Adobe’s Photoshop (PS). This is one of the most amazing tools in photography and it is step I never skip. Once in PS I remove the pin using the clone stamp tool or as part of a background replacement, I use the clone stamp tool or spot healing brush to remove foreign bodies or dirt that missed during cleaning. I occasionally dodge and burn sparingly, to accentuate or de-emphasize certain features. In this case I also used the clone stamp tool to remove a little of the residual haze from the forelimb halo and also removed a tiny amount haze from the left middle appendage in the are of the knee. This is the image saved back into LR.
The final image - “Galloping Weevil” (with one missing “foot” mysteriously rediscovered)
And that is all I have to say about the dreaded halo artifact, except maybe that I don’t really dread it that much these days. It is a part of focus stacking and as long as we keep in mind why it is there and take steps to avoid it becoming the most prominent part of our photograph, it really shouldn’t be a source of too much dread. There are other artifacts, some of which look a little like halo, that require a different approach, but if you get this one down, you will have tackled the worst of it. The knowledge is useful, but the improvement will come from diligent and mindful practice. It takes work and it takes time, but it is oh so worth it!
¹ This article addresses halo correction using Zerene Stacker. Other stacking programs offer some of the tools needed to effectively eliminate this annoying artifact, but only Zerene gives you all of them.
I hope this was helpful and gives you some new ideas to try out in your own workflow, but if you need some more focused assistance, get in touch by leaving me a message here or on my contact page. I do offer private training to those who feel they have got as far as they can on their own and need a little help to get to the next level. This stuff is not easy and todays topic is one of those areas that lends itself particularly well to one-on-one tutoring. I usually stay booked up but have a couple of openings that will soon be avaiable. For more information, get in touch.